feat: Updates to AI components for additional chat prompt use cases#10335
Conversation
other changes like gettring rid of the ToggleButtonGroup are deffered in favor of wrapping the group in a Toolbar
…arent using our prompt field
…ve a panel to display
…ursor positioning when token added via + menu also support escape /click on stop button to stop streaming in chat story
this allows a callback (aka like "compact") command to clear the user partial filter text and close the autocomplete menu in the field
| } from '../src/PromptField'; | ||
| export {ResponseStatus, ResponseStatusTitle, ResponseStatusPanel} from '../src/ResponseStatus'; | ||
| export {Chat, Thread, ThreadItem, ThreadScrollButton} from '../src/Chat'; | ||
| export {Chat, Thread, ThreadItem, ThreadScrollButton, PromptFocusContext} from '../src/Chat'; |
There was a problem hiding this comment.
exported in case a user don't use our PromptField with Chat. They will need to wire up the onFocusChange to their input field
There was a problem hiding this comment.
strings to be translated, open to discussion for copy
| borderRadius: 'default' | ||
| }); | ||
|
|
||
| const attachmentErrorStyles = style({ |
There was a problem hiding this comment.
The styles for invalid + thumbnail only aren't added yet, to be discussed with design
| // TODO: mirrors tokenfield, maybe should also be a generic too | ||
| value?: TokenSegmentList; | ||
| defaultValue?: TokenSegmentList; | ||
| onChange?: (value: TokenSegmentList) => void; | ||
| // TODO: discuss, I can imagine a case where we also want to prefill these | ||
| attachments?: PromptFieldAttachment[]; | ||
| defaultAttachments?: PromptFieldAttachment[]; | ||
| onAttachmentsChange?: (attachments: PromptFieldAttachment[]) => void; |
| if (text.length === 0) { | ||
| return [{type: 'text', text}]; | ||
| } | ||
| function tokenizeURLs(text: string): TokenFieldSegment[] { |
There was a problem hiding this comment.
change here looks significant but its just moving/extracting some logic since I had to update AutoLinkingSegmentList to auto convert urls into tokens if added programmatically via controlled value
| let [prompt, setPrompt] = useState<TokenSegmentList>(new AutoLinkingSegmentList([])); | ||
| let [attachments, setAttachments] = useState<PromptFieldAttachment[]>([]); | ||
| let stringFormatter = useLocalizedStringFormatter(intlMessages, '@react-spectrum/ai'); | ||
| let [prompt, setPrompt] = useControlledState( |
There was a problem hiding this comment.
as mentioned before, adds support for controlled value in the PromptField for things like "click this external button to pre-fill a common prompt into your field"
| children?: (segment: TokenSegment) => React.ReactElement; | ||
| pixelLoader?: Cell[] | Cell[][]; | ||
| placeholder?: string; | ||
| onKeyDown?: (e: React.KeyboardEvent<HTMLDivElement>) => void; |
There was a problem hiding this comment.
support for custom user defined keyboard commands (Opt + Enter for followup while streaming, arrow keys to autofil old prompts)
| setCursor(inputRef.current, position); | ||
| // TODO: double check this, claude debugged this one, but essentially reproduced with plain text insertion commands | ||
| // triggered one after another | ||
| // focus() fires a synchronous selectionchange before setCursor can set | ||
| // isProgrammaticSelectionChange, which resets caretPosition to {0,0} in | ||
| // TokenField's useSelectionChange handler. Re-assert the correct position. | ||
| setPrompt(value => value.withCaretPosition(position)); |
There was a problem hiding this comment.
as mentioned in the comment, seems to be a race here that caused inconsistent text cursor positioning when doing insertions via the "+" menu.
There was a problem hiding this comment.
interesting, we unset all the blocking and trapping when dialogs close, wonder why this was causing an issue with the cursor positioning
do you know what it was racing against? was it restore focus from the dialog to the trigger? or what was happening?
There was a problem hiding this comment.
yeah so the changes are kinda two fold. The addition of setCursor is to fix the issue where injecting a token into the field via the "+" menu is just putting the cursor at the start of the field, which happens because the token injection happens when focus is still in the "+" menu when it happens, but
react-spectrum/packages/@react-spectrum/ai/src/TokenField.tsx
Lines 181 to 184 in 5b99560
The setPrompt race is a bit more complicated, and that is to fix a flow like "inject /feedback via menu -> same for /btw afterwards -> inject /feedback again" where then last /feedback is inserted into the wrong place. What seems to happen is that two separate cursor updates happen due to the inputRef.current.focus(); and setCursor here, and the inputRef.current.focus(); messes up the cursor position due to the isProgrammaticSelectionChange flag being flipped to false after setCursor is called, thus allowing the internally tracked caret position to get set to 0 erroneously instead of preserving . I think ideally isProgrammaticSelectionChange should remain as true across both the .focus and the setCursor so that we preserve our calculated caret position but that is a bit tricky and is easier to just reupdate to the right position
| /** | ||
| * Whether the response is still being generated. When true, a ProgressCircle replaces | ||
| * the chevron and the panel cannot be expanded. The trigger remains focusable. | ||
| * The current status of the response. |
There was a problem hiding this comment.
API change we'll have to call out
There was a problem hiding this comment.
based off what Coworker had, but refactored a bit since it didn't seem they actually needed some of the callbacks
There was a problem hiding this comment.
see https://webaudio.github.io/web-speech-api/ and https://developer.mozilla.org/en-US/docs/Web/API/SpeechRecognition, mostly generated by Claude but I did a spot check against the above
|
Build successful! 🎉 |
|
Build successful! 🎉 |
| */ | ||
|
|
||
| export {useTokenField, positionToDOMRange} from '../src/tokenfield/useTokenField'; | ||
| export {useTokenField, positionToDOMRange, setSelection} from '../src/tokenfield/useTokenField'; |
There was a problem hiding this comment.
note the addition export here, required since I was using setCursor to fix some cursor positioning issues
|
Build successful! 🎉 |
|
Build successful! 🎉 |
|
Build successful! 🎉 |
|
Build successful! 🎉 |
| }, [transcript, isVoiceListening]); | ||
|
|
||
| useEffect(() => { | ||
| if (isDisabled && isVoiceListening) { |
There was a problem hiding this comment.
should isDisabled be passed into useVoiceInput? that's a pretty common pattern
There was a problem hiding this comment.
makes sense, I can see about adding that to the hook
There was a problem hiding this comment.
I will do this as a followup, gonna try and get this up as a nightly ASAP for now
| }; | ||
|
|
||
| // logic for using up/down arrow keys to fill field with previous prompts | ||
| let onKeyDown = (e: React.KeyboardEvent<HTMLDivElement>) => { |
There was a problem hiding this comment.
Are we sure we want this? I thought we were not going to do it this way and implement a menu instead?
There was a problem hiding this comment.
for the prev/next prompt cycling via up/down arrow, yes we are going to suggest doing it with the menu but this was more of a proof of concept in the story that users could still implement that functionality if they wanted to. onKeyDown is still needed to handle custom keyboard shortcuts like Option+Enter to submit a followup prompt as a response is streaming and/or handling for Enter for "steering"
|
Build successful! 🎉 |
## API Changes
react-aria-components/react-aria-components:TokenField TokenField <T extends TokenFieldValue = TokenFieldValue> {
allowsNewlines?: boolean
aria-describedby?: string
aria-details?: string
aria-label?: string
aria-labelledby?: string
autoFocus?: boolean
children?: ChildrenOrFunction<TokenFieldRenderProps>
className?: ClassNameOrFunction<TokenFieldRenderProps> = 'react-aria-TokenField'
defaultValue?: TokenFieldValue
isDisabled?: boolean
isReadOnly?: boolean
onBlur?: (FocusEvent<Target>) => void
onChange?: (TokenFieldValue) => void
onCopy?: ClipboardEventHandler<TokenFieldValue>
onCut?: ClipboardEventHandler<TokenFieldValue>
onFocus?: (FocusEvent<Target>) => void
onFocusChange?: (boolean) => void
- onKeyDown?: (KeyboardEvent) => void
- onKeyUp?: (KeyboardEvent) => void
+ onKeyDown?: (React.KeyboardEvent<HTMLDivElement>) => void
+ onKeyUp?: (React.KeyboardEvent<HTMLDivElement>) => void
onPaste?: ClipboardEventHandler<TokenFieldValue>
onSubmit?: () => void
render?: DOMRenderFunction<keyof React.JSX.IntrinsicElements, TokenFieldRenderProps>
role?: 'textbox' | 'searchbox' | 'combobox' = 'textbox'
style?: StyleOrFunction<TokenFieldRenderProps>
value?: TokenFieldValue
}/react-aria-components:TokenFieldProps TokenFieldProps <T extends TokenFieldValue = TokenFieldValue> {
allowsNewlines?: boolean
aria-describedby?: string
aria-details?: string
aria-label?: string
aria-labelledby?: string
autoFocus?: boolean
children?: ChildrenOrFunction<TokenFieldRenderProps>
className?: ClassNameOrFunction<TokenFieldRenderProps> = 'react-aria-TokenField'
defaultValue?: TokenFieldValue
isDisabled?: boolean
isReadOnly?: boolean
onBlur?: (FocusEvent<Target>) => void
onChange?: (TokenFieldValue) => void
onCopy?: ClipboardEventHandler<TokenFieldValue>
onCut?: ClipboardEventHandler<TokenFieldValue>
onFocus?: (FocusEvent<Target>) => void
onFocusChange?: (boolean) => void
- onKeyDown?: (KeyboardEvent) => void
- onKeyUp?: (KeyboardEvent) => void
+ onKeyDown?: (React.KeyboardEvent<HTMLDivElement>) => void
+ onKeyUp?: (React.KeyboardEvent<HTMLDivElement>) => void
onPaste?: ClipboardEventHandler<TokenFieldValue>
onSubmit?: () => void
render?: DOMRenderFunction<keyof React.JSX.IntrinsicElements, TokenFieldRenderProps>
role?: 'textbox' | 'searchbox' | 'combobox' = 'textbox'
style?: StyleOrFunction<TokenFieldRenderProps>
value?: TokenFieldValue
}@react-spectrum/ai/@react-spectrum/ai:Attachment Attachment {
allowsArrowNavigation?: boolean
aria-describedby?: string
aria-details?: string
aria-label?: string
aria-labelledby?: string
children: ReactNode | (AttachmentRenderProps) => ReactNode
density?: 'compact' | 'regular' | 'spacious' = 'regular'
download?: boolean | string
focusMode?: 'child' | 'row'
href?: Href
hrefLang?: string
id?: Key
isDisabled?: boolean
+ isInvalid?: boolean
onAction?: () => void
onPress?: (PressEvent) => void
onPressChange?: (boolean) => void
onPressEnd?: (PressEvent) => void
onPressUp?: (PressEvent) => void
ping?: string
referrerPolicy?: HTMLAttributeReferrerPolicy
rel?: string
render?: DOMRenderFunction<keyof React.JSX.IntrinsicElements, TagRenderProps>
routerOptions?: RouterOptions
size?: 'XS' | 'S' | 'M' | 'L' | 'XL' = 'M'
styles?: StyleString
target?: HTMLAttributeAnchorTarget
textValue?: string
uploadProgress?: number
value?: T
variant?: 'primary' | 'secondary' | 'tertiary' | 'quiet' = 'primary'
}/@react-spectrum/ai:MessageFeedback MessageFeedback {
aria-describedby?: string
aria-details?: string
aria-label?: string
aria-labelledby?: string
defaultValue?: MessageFeedbackValue
id?: string
isDisabled?: boolean
onChange?: (MessageFeedbackValue) => void
+ size?: 'XS' | 'S' | 'M' | 'L' | 'XL' = 'M'
slot?: string | null
styles?: StylesPropWithHeight
thumbDownLabel?: string
thumbUpLabel?: string
}/@react-spectrum/ai:PromptField PromptField {
acceptedAttachmentTypes?: Array<string>
+ attachments?: Array<PromptFieldAttachment>
children: React.ReactNode
+ defaultAttachments?: Array<PromptFieldAttachment>
+ defaultValue?: TokenFieldValue
isGenerating?: boolean
onAddAttachments?: (Array<PromptFieldAttachment>) => void
+ onAttachmentsChange?: (Array<PromptFieldAttachment>) => void
+ onChange?: (TokenFieldValue) => void
onRemoveAttachments?: (Array<PromptFieldAttachment>) => void
onStop?: () => void
onSubmit?: (TokenFieldValue, Array<PromptFieldAttachment>) => void
styles?: StyleString
+ value?: TokenFieldValue
variant?: 'balanced' | 'prominent' | 'subtle'
}/@react-spectrum/ai:PromptTokenField PromptTokenField {
children?: (TokenSegment) => React.ReactElement
completionTrigger?: RegExp
+ onKeyDown?: (React.KeyboardEvent<HTMLDivElement>) => void
pixelLoader?: Array<Cell> | Array<Array<Cell>>
placeholder?: string
renderCompletions?: (string) => Array<React.ReactNode> | null | Promise<Array<React.ReactNode> | null>
}/@react-spectrum/ai:ResponseStatus ResponseStatus {
children: ReactNode
defaultExpanded?: boolean
density?: 'compact' | 'regular' | 'spacious' = 'regular'
id?: Key
isDisabled?: boolean
isExpanded?: boolean
- isLoading?: boolean
onExpandedChange?: (boolean) => void
size?: 'S' | 'M' | 'L' | 'XL' = 'M'
slot?: string | null
+ status?: 'loading' | 'failed' | 'success' = 'loading'
styles?: StyleString
}/@react-spectrum/ai:ThreadItem ThreadItem {
allowsArrowNavigation?: boolean
children?: ChildrenOrFunction<GridListItemRenderProps>
focusMode?: 'child' | 'row'
+ id?: Key
isStreaming?: boolean
shouldAnnounceOnMount?: boolean
styles?: StyleString
textValue?: string/@react-spectrum/ai:AttachmentProps AttachmentProps {
allowsArrowNavigation?: boolean
aria-describedby?: string
aria-details?: string
aria-label?: string
aria-labelledby?: string
children: ReactNode | (AttachmentRenderProps) => ReactNode
density?: 'compact' | 'regular' | 'spacious' = 'regular'
download?: boolean | string
focusMode?: 'child' | 'row'
href?: Href
hrefLang?: string
id?: Key
isDisabled?: boolean
+ isInvalid?: boolean
onAction?: () => void
onPress?: (PressEvent) => void
onPressChange?: (boolean) => void
onPressEnd?: (PressEvent) => void
onPressUp?: (PressEvent) => void
ping?: string
referrerPolicy?: HTMLAttributeReferrerPolicy
rel?: string
render?: DOMRenderFunction<keyof React.JSX.IntrinsicElements, TagRenderProps>
routerOptions?: RouterOptions
size?: 'XS' | 'S' | 'M' | 'L' | 'XL' = 'M'
styles?: StyleString
target?: HTMLAttributeAnchorTarget
textValue?: string
uploadProgress?: number
value?: T
variant?: 'primary' | 'secondary' | 'tertiary' | 'quiet' = 'primary'
}/@react-spectrum/ai:PromptFieldProps PromptFieldProps {
acceptedAttachmentTypes?: Array<string>
+ attachments?: Array<PromptFieldAttachment>
children: React.ReactNode
+ defaultAttachments?: Array<PromptFieldAttachment>
+ defaultValue?: TokenFieldValue
isGenerating?: boolean
onAddAttachments?: (Array<PromptFieldAttachment>) => void
+ onAttachmentsChange?: (Array<PromptFieldAttachment>) => void
+ onChange?: (TokenFieldValue) => void
onRemoveAttachments?: (Array<PromptFieldAttachment>) => void
onStop?: () => void
onSubmit?: (TokenFieldValue, Array<PromptFieldAttachment>) => void
styles?: StyleString
+ value?: TokenFieldValue
variant?: 'balanced' | 'prominent' | 'subtle'
}/@react-spectrum/ai:PromptTokenFieldProps PromptTokenFieldProps {
children?: (TokenSegment) => React.ReactElement
completionTrigger?: RegExp
+ onKeyDown?: (React.KeyboardEvent<HTMLDivElement>) => void
pixelLoader?: Array<Cell> | Array<Array<Cell>>
placeholder?: string
renderCompletions?: (string) => Array<React.ReactNode> | null | Promise<Array<React.ReactNode> | null>
}/@react-spectrum/ai:MessageFeedbackProps MessageFeedbackProps {
aria-describedby?: string
aria-details?: string
aria-label?: string
aria-labelledby?: string
defaultValue?: MessageFeedbackValue
id?: string
isDisabled?: boolean
onChange?: (MessageFeedbackValue) => void
+ size?: 'XS' | 'S' | 'M' | 'L' | 'XL' = 'M'
slot?: string | null
styles?: StylesPropWithHeight
thumbDownLabel?: string
thumbUpLabel?: string
}/@react-spectrum/ai:ResponseStatusProps ResponseStatusProps {
children: ReactNode
defaultExpanded?: boolean
density?: 'compact' | 'regular' | 'spacious' = 'regular'
id?: Key
isDisabled?: boolean
isExpanded?: boolean
- isLoading?: boolean
onExpandedChange?: (boolean) => void
size?: 'S' | 'M' | 'L' | 'XL' = 'M'
slot?: string | null
+ status?: 'loading' | 'failed' | 'success' = 'loading'
styles?: StyleString
}/@react-spectrum/ai:ThreadItemProps ThreadItemProps {
allowsArrowNavigation?: boolean
children?: ChildrenOrFunction<GridListItemRenderProps>
focusMode?: 'child' | 'row'
+ id?: Key
isStreaming?: boolean
shouldAnnounceOnMount?: boolean
styles?: StyleString
textValue?: string/@react-spectrum/ai:AutoLinkingTokenFieldValue+AutoLinkingTokenFieldValue {
+ caretPosition: Position
+ constructor: (readonly Array<TokenFieldSegment<T>>, TokenFieldValueOptions) => void
+ delete: (Position, Intl.Segmenter, any, any) => this
+ deleteLine: (Position, any, any) => this
+ endCoalescing: () => void
+ findBoundaryWithSegmenter: (Position, Intl.Segmenter, any) => Position | null
+ findLineBoundary: (Position, any) => Position | null
+ findText: (Position, any, string | RegExp) => Position | null
+ redo: () => this
+ replaceRange: (Position, Position, string, any) => this
+ replaceRangeWithSegments: (Position, Position, Array<TokenFieldSegment<T>>, any) => this
+ segments: readonly Array<TokenFieldSegment<T>>
+ slice: (Position, Position) => this
+ toString: () => string
+ tokenize: (string) => Array<TokenFieldSegment>
+ undo: () => this
+ withCaretPosition: (Position) => this
+}/@react-spectrum/ai:CommandMenuItem+CommandMenuItem {
+ UNSAFE_className?: UnsafeClassName
+ UNSAFE_style?: CSSProperties
+ aria-label?: string
+ children: ReactNode
+ download?: boolean | string
+ href?: Href
+ hrefLang?: string
+ id?: Key
+ isDisabled?: boolean
+ onAction?: () => void
+ onBlur?: (FocusEvent<Target>) => void
+ onFocus?: (FocusEvent<Target>) => void
+ onFocusChange?: (boolean) => void
+ onHoverChange?: (boolean) => void
+ onHoverEnd?: (HoverEvent) => void
+ onHoverStart?: (HoverEvent) => void
+ onPress?: (PressEvent) => void
+ onPressChange?: (boolean) => void
+ onPressEnd?: (PressEvent) => void
+ onPressStart?: (PressEvent) => void
+ onPressUp?: (PressEvent) => void
+ ping?: string
+ referrerPolicy?: HTMLAttributeReferrerPolicy
+ rel?: string
+ routerOptions?: RouterOptions
+ shouldCloseOnSelect?: boolean
+ styles?: StylesProp
+ target?: HTMLAttributeAnchorTarget
+ textValue?: string
+ value?: T
+}/@react-spectrum/ai:InsertTextMenuItem+InsertTextMenuItem {
+ UNSAFE_className?: UnsafeClassName
+ UNSAFE_style?: CSSProperties
+ aria-label?: string
+ children: ReactNode
+ download?: boolean | string
+ href?: Href
+ hrefLang?: string
+ id?: Key
+ isDisabled?: boolean
+ onAction?: () => void
+ onBlur?: (FocusEvent<Target>) => void
+ onFocus?: (FocusEvent<Target>) => void
+ onFocusChange?: (boolean) => void
+ onHoverChange?: (boolean) => void
+ onHoverEnd?: (HoverEvent) => void
+ onHoverStart?: (HoverEvent) => void
+ onPress?: (PressEvent) => void
+ onPressChange?: (boolean) => void
+ onPressEnd?: (PressEvent) => void
+ onPressStart?: (PressEvent) => void
+ onPressUp?: (PressEvent) => void
+ ping?: string
+ referrerPolicy?: HTMLAttributeReferrerPolicy
+ rel?: string
+ routerOptions?: RouterOptions
+ shouldCloseOnSelect?: boolean
+ styles?: StylesProp
+ target?: HTMLAttributeAnchorTarget
+ textValue?: string
+ value?: T
+}/@react-spectrum/ai:PromptFocusContext+PromptFocusContext {
+ UNTYPED
+} |
Agent Skills ChangesModified (5)
InstallReact Spectrum S2: React Aria: |
fills in gaps in our current AI components that we found when integrating with other experiences. Includes the following:
✅ Pull Request Checklist:
📝 Test Instructions:
See the description above for all the added features. Test the AttachmentList, Chat, and PromptField stories and verify behavior like invalid attachments, token/callback/plain text injection in the PromptField, voice input support, etc
🧢 Your Project:
RSP